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Abstract — We study the interaction between a network 
designer and an adversary over a dynamical network. The 
network consists of nodes performing continuous-time dis- 
tributed averaging. The goal of the network designer is to 
assist the nodes reach consensus by changing the weights of 
a limited number of links in the network. Meanwhile, an 
adversary strategically disconnects a set of links to prevent the 
nodes from converging. We formulate two problems to describe 
this competition where the order in which the players act is 
reversed in the two problems. We utilize Pontryagin's Maximum 
Principle (MP) to tackle both problems and derive the optimal 
strategies. Although the canonical equations provided by the 
MP are intractable, we provide an alternative characterization 
for the optimal strategies that highlights a connection with 
potential theory. Finally, we provide a sufficient condition for 
the existence of a saddle-point equilibrium (SPE) for this zero- 
sum game. 

I. Introduction 

Various physical and biological phenomena where global 
patterns of behaviour stem from local interactions have been 
modelled using linear averaging dynamics. In such dynamics 
an agent updates its value as a linear combination of the 
values of its neighbors. Averaging dynamics is the basic 
building block in many multi-agent systems, and it is widely 
used whenever an application requires multiple agents, who 
are graphically constrained, to synchronize their measure- 
ments. Examples include formation control, coverage, dis- 
tributed estimation and optimization, and flocking [l]-[3]. 
Besides engineering, linear averaging finds applications in 
other fields. For instance, social scientists use averaging to 
describe the evolution of opinions in networks [4]. 

In practice, communication among agents is prone to 
different non-idealities which can affect the convergence 
properties of distributed algorithms. Transmission delays [5], 
noisy links [6], [7], and quantization [8] are some examples 
of non-idealities that are due to the physical nature of the 
application. In addition to physical restrictions, researchers 
have also studied averaging dynamics in the presence of 
malicious nodes in the network [9], [10]. Various algorithms 
that guarantee resilience against node failures have been 
proposed in the literature [11]. 

In [12], we explored the effect of an external adversary 
who attempts to prevent the nodes from reaching consensus 
by disconnecting certain links in the network. We derived his 
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optimal strategy and demonstrated that it admits a potential- 
theoretic analogy. Here, we introduce a network designer that 
attempts to counter the effect of the adversary and help the 
nodes reach consensus by changing the weights of certain 
links. Both the adversary and the designer are constrained by 
their physical properties, e.g., battery life and communication 
range. To capture such constraints, we allow the adversary 
and the designer to affect only a fixed number of links. 

Such an interaction between a network designer and an 
adversary can occur in various practical applications. For 
example, in a wireless network, the adversary can be a 
jammer who is capable of breaking links by injecting high 
noise signals that disrupt the communication among nodes. 
The link weights in such a network represent the capacities 
of the corresponding links. The designer can modify the 
capacity of a certain link using various communication 
techniques such as introducing parallel channels between two 
nodes as in orthogonal frequency division multiple access 
(OFDMA) networks [13]. 

Our model is different from the current literature in two 
ways: (i) the adversary and the designer compete over a 
dynamical network. This is different from the problems 
studied in the computer science and economics communities 
where the network is usually static [14]; (ii) the players in 
our model are constrained and cannot operate with an infinite 
budget. This enables us to model practical scenarios more 
closely rather than allowing the malicious behaviour to be 
unrestricted as in [9], [15], [16]. 

The main goal of this work is to derive optimal strategies 
for the designer and the adversary who have conflicting 
objectives. Because the order in which the players act affects 
the resulting utilities, we formulate two problems based on 
the order of play, allowing each player to have the first-move- 
advantage in one of the problems. When the adversary is 
allowed to play first, he is capable of restricting the available 
actions of the designer since some links will disappear from 
the network. Hence, if we were to cast a zero-sum game 
between the players, one should not expect the existence 
of a saddle-point equilibrium (SPE) in pure strategies. The 
question we would like to answer is then: are there scenarios 
where the order of play does not affect the utilities of the 
players, which leads to the existence of an SPE? 

Accordingly, the contributions of this paper are as follows. 

• We capture the interaction between the designer and the 
adversary by formulating two problems. In the min- 
max problem, the designer declares a strategy first to 
which the adversary reacts by its optimal response. The 
second problem is a max-min one where the order of 
play is reversed. We derive the optimal strategies in both 



problems by employing the Maximum Principle (MP). 

• We provide a method to compute the optimal strategies 
without requiring the adjoint equations to be solved. 
This method provides a new characterization for the 
optimal strategies in terms of potential-theoretic quan- 
tities. 

• We derive a sufficient condition guaranteeing the exis- 
tence of an SPE. 

The rest of the paper is organized as follows. In Section 
|IT| we formulate and provide the preliminaries of the min- 
max and max-min problems. We analyze both problems in 
Section [TIT] and derive their solutions. An implementation 



method is provided in Section IV We provide a sufficient 
condition for the existence of an SPE in Section |VJ and we 
conclude the paper in Section [VI] An Appendix contains two 
lemmas used in the derivation of the sufficient condition. 

Notation: We denote the set of edges in a graph Q by £{G). 
When clear from the context, we will drop the argument of 
any set defined on a graph. To emphasize the effect of link 
removal by the adversary, we will sometimes write Q(u{t)) 
to denote the graph resulting after the adversary acts at time 
t. We will use £,>*(■) to mean YTj=2 £i=i CO- 
IL Problem Description 

Consider a connected network of n nodes described by an 
undirected graph Q with n vertices and m edges. We will 
denote an edge in £{G) between nodes i and j by {i, j}. 
The value, or state, of the nodes at time instant t is given 
by x(t) = [xi(t), ...,x n (t)] T . The nodes start with an initial 
value x(0) = xq, and they are interested in computing the 
average of their initial measurements x avg = ^ J27=i x i(®) 
via local averaging. We consider the continuous -time aver- 
aging dynamics given by 



x(t) = A(t)x(t), x(0) = x , 
where the matrix A(t) has the following properties 

A(t) = A(t) T , Aij(t)>0 i^j, A(t)l = 0. 



(1) 



Define x = lx avg and let M = ^^. A well-known result 
states that, given the above assumptions, the nodes will 
reach consensus as t —> oo, i.e., \im t ^ocx(t) = x [1]. 
In the absence of the adversary and the network designer, 
A(t) is time-invariant with Aij(t) = a^. To achieve their 
respective objectives, the designer and the adversary control 
the elements of A(t) as we explain next. 

The adversary attempts to slow down convergence by 
breaking at most £ < m links at each time instant. 
Let Uij(t) G {0,1} be the weight the adversary as- 
signs to link {i,j}. He breaks link {i,j} at time t 
when Uij(t) = 1. His control is given by u(t) = 
[u 12 (t),ui 3 (t),...,u ln (t),U23(t),---,U( n _ 1)n (t)} T . We will 
denote the number of links the adversary breaks at time t 
by N u (t). In accordance with the above, the strategy space 
of the adversary is 

U = \ue {0,1}(5) :N u <i}. 



Meanwhile, the network designer attempts to accelerate 
convergence by controlling the weights of the edges. Let 
v(t) = [v 12 (t),v 13 (t), ...,v ln (t),v 2 s(t), ...,v {n _ 1)n (t)] T be 
the control of the designer. The designer can change the 
weight of a given link by adding vij (t) to its weight a^ . We 
assume that Vij(t) G [0, b] and that the number of links the 
designer modifies N v (t) is at most L The strategy set of the 
designer is therefore 

V= [ve [0,6] (2) :N V <f}. 

If {i, j} ^ £, then uij{t) = Vij(t) = for all t. Given the 
above definitions, we can write the {z, j}-th element of the 
matrix A(u(i),v(i)) as 

A i;j (u(t),v(t)) = (dij +Vij(t)) • (1 - Uij{t)). 

Implicit in this definition is the assumption that the effect 
of the adversary and the designer on the links lasts only 
instantly, and the weights of the links are reset to a^ at the 
next time instant. This is motivated by applications where 
the state of a channel deteriorates only when the adversary 
is acting, e.g., wireless communications where the effect of 
the jammer is not permanent. 
Define the following functional: 



J(u,v) 



f k(t)\ 

Jo 



<t) 



x\ dt, 



where the kernel k(t) is positive and integrable over [0,T]. 
The utility of the adversary is then given by J(u,v), and 
that of the designer is —J(u,v). We will study the following 
two problems. In the first problem, the adversary acts first 
by selecting the links he is interested in breaking. Then, the 
network designer optimizes his choices over the resulting 
graph Q{u(t)). More formally, we have 

max min J(u, v) 
u(t)eUv(t)eV 



s.t. 



x(t) = A(u(t), v(t))x(t), x(0) = x . 



In the second problem, the order is reversed, and the 
designer acts first: 



min max J(u, v) 

v(t)evu(t)eu 



s.t. 



x(t) = A(u(i), v(t))x(t), x(0) = x . 



In a computer network, where consensus could model the 
spread of a virus, the max-min problem allows the network 
architect (who is the maximizer here) to implement networks 
that are robust against a strategic virus diffusion. The min- 
max problem finds applications in army combat situations 
where the designer (the minimizer) attempts to counter the 
attacks of the enemy to disrupt the communications among 
agents. In both problems, we make the following assumption: 

Assumption 1: The initial matrix A(0,0), the state x(t), 
and the value b are common information to both players. 
The following two remarks are in order. 

Remark 1: (Problem Complexity) Let us consider the 
problem of the adversary for a given strategy of the designer. 



Assume that the adversary can act in K time instances over 
the interval [0, T]. Then, for small £, the total number of 
links that need to be tested in a brute-force approach is 

r \ K 



m 



(2) 



Clearly, the brute-force approach incurs exponential-time 
complexity, especially if any of m, £, k becomes large. The 
same argument applies to the network designer. 

Remark 2: (Non-Rectangular Strategies) When the strat- 
egy sets are rectangular, i.e., the strategy of one player does 
not restrict the strategy space of the other, the following holds 



V_ — max min J(u. v) < min max J(u, v) = V, 

u{t)euv{t)ev v(t)evu(t)eu 

(3) 
where V_, V are called the lower and upper values of the 
game, respectively. An SPE in pure strategies exists when 
V = V. When the strategy spaces are non-rectangular, one 
should not expect the existence of an SPE in pure-strategies 
nor the order in ([3]). In the max-min problem, the adversary 
restricts the strategy space of the designer since u(t) removes 
links from Q. Hence, the max-min and min-max problems 
should be studied separately. 

III. Adversary vs. Network Designer: Two 
Problems 

We will now present the solutions for the two problems 
presented above. To arrive at the optimal strategies of the 
players, we employ the MP. In Section |IV| we prove that 
N* = N* = ^; we will use this fact in the proofs of this 
section. In what follows, we will often drop the time index 
and other arguments for notational simplicity. 

A. The Min-Max Problem 

The Hamiltonian associated with the max-min problem is: 

H(x,p,u,v) = k(t)\x(t) -x\ 2 +p T (t)A(t)x(t). 

The first-order necessary conditions for optimality are: 
d 



P 
x 



dx 



H 



-2k(x -x)- Ap, p(T) = (4) 

Ax, x(0) = x (5) 

arg max arg min {H : u E U, v E V} , 

where x,p E C 1 [0,T], the space of continuously differen- 
tiable functions over [0,T]. 

To find the optimal strategies, let us first write 



p T Ax 



i=l 







Pt 



7 j AijXi 



7 j AijXj 






j=lj& 3 = 1,3^1 

iJ^ij + v#)(l - Uij)(Pj ~Pi)(Xi ~ Xj). 
j>i 



Define the function 



and let us write 

max min H 

ueu vev 



(Pj ~Pi)(Xi ~Xj), 



max min k \x 

ueu vev 

k\x — x\ 



-p Ax 



^^E^"- (6) 



ueu vev 



j>i 

Note that we cannot decouple the maximization (or mini- 
mization) into (g) maximization problems, each correspond- 
ing to a link or a single term inside the double summation. 
This is due to the constraint on the number of links that 
can be targeted by the players. Before finding the optimal 
strategies, let us first define the following sets. Let F(G r ) = 
{fij < : {ij} E £{Q')} for some graph Q' . It should 
be noted that although {i,j} and {j, i} belong to £(G f ), 
we only include fy once in J r (^ / ), i.e., we treat {i : j} 
and {j, i} as one link. This applies to all the definitions 
to follow. Let FniG') be a ranking of the set F(G') such 
that J;,i(^) < ... < ^,|^)|(£'). Further, let T^G') 
be the set containing the smallest £ values in J r 7r (^ / ). It is 
understood that if ^(G') contains fewer than £ negative 
elements, T^ will contain all those elements. Define the 
set operator §i : S(G') —* £{G') that returns the links 
in £{G') that correspond to the elements of S(G')- Also, 
define $* : S[G') -> £(G') that returns the links in £((?') 
corresponding to the smallest i elements of the set S{G'). 
When \S(G')\ < i, *i(5(5 ; )) = *(5(S ; ))« We also ado Pt 
the convention ^o(-) — {0}- We can then write 

J*(Q') = *t ({fij < ^ M1 (G f ) : {i,j} E £{G f )}) . 

Define the sets V X {G') = {a^fa < : {ij} E £(G')}, 
V 2 (G f ) = {(aij + b)fij < : {i,j} E £(G')}, and V(G') = 
Vi(G') U V 2 (G'). Also, let V n (G f ) be a ranking of V(G') 
such that V^iiG') < • • • < ^-k,\v{Q')\{G'). Finally, define 

Vi{G') = M{^ijfi3<^i + i(G f )or 

(a^ + b)fij < V n/+1 (G f ) : {ij} E £(G')}). 

The following theorem specifies the optimal strategies of 
the adversary and the designer. 

Theorem 1: The optimal strategies for the network de- 
signer and the adversary in the max-min problem are 

b, {i,j}efi(g*) 
««(*) - { 0, {i,j} <£J*(g*) or f tj >o 

[0,6], otherwise 

1, {i,j}eVi(G) 

<(*) - <( 0, {i,j}^Vi(G)orf ij >0 

{0, 1}, otherwise 

where G* = G(u*) is the graph resulting after the adversary 
applies u*. 

Proof: For both strategies, when fy = 0, the MP does 
not tell us what the optimal action is. Hence, both players 
can select any action from their strategy spaces in this case, 
provided they do not exceed their budget constraints. Let us 



start by deriving v*. By ([6]), the terms a^faj do not affect 
the minimization problem. Also, note that Q* contains the 
links for which u\, = only. Then, we can write 



axgrninj^ Aij fi 



13 



argmm 

vev 



{ij}e£(G*) 



Now consider the sum on the right hand side. Being a 
minimizer, the designer should not amplify the positive faj's; 
hence, he must set v*j = when faj > 0. Then, by recalling 
the definition of J 7 ^ ((/*), we obtain 



Yl v ijfr 



/ j V ijJij H" / j v ijJi: 



{ij}es(g*) 



j>i j>i 



j>i 

{i,j}eri(G*) 
Note that the last inequality can indeed be achieved because 
\J r£ (G*)\ < f~ Hence, the optimal v* is as claimed. 

In this problem, the adversary has the first-mover- 
advantage and needs to dispose of the links that can reduce 
his utility. Because the adversary attempts to maximize the 
minimum of p T Ax, it follows that u*. = when faj > 0. Let 
us now consider the links with faj < 0. The adversary knows 
that the designer attempts to make the negative faj's smaller 
by adding b to the corresponding edge weights. However, 
we cannot rule out the possibility that (a^ + b)fak > a>ijfij. 
Hence, the adversary is not only interested in finding the 
smallest negative (a^ + b)faj's, but also needs to consider 
the dijfij's themselves. It follows that the adversary needs 
to find the terms that can become very small (negative) and 
set Uij = 1 to the corresponding links. But those links are 
exactly the ones included inV^(Q). Formally, and using the 
notation A*j — (a^ +v*j)(l — u^), we have 



J>t 



J>1 
{iJ}evi(g) 



{iJWvUG) 



< E faj+vtyfiji 



where the last inequality follows from the fact that faj < 
for all {i,j} G V^(Q). This confirms that u* is as claimed. 

■ 

B. The Min-Max Problem 

We now study the min-max problem. Similar to the above, 
we can write 



min max H 

vev ueu 



k \x ■ 



mm max > A^ fa , 

vev ueu *• — ' 



Hj J 13 • 



We again note that one needs to be careful to observe that the 
minimization (or maximization) problem does not decouple 
due to the constraint on the controls of the two players. 
Although the adversary's strategy can be derived using an 
approach similar to the previous problem, the designer's 
strategy requires more care. Let C(v) = {{a>ij + v ij)fij < 



TABLE I 
Algorithm I: Computing the optimal strategy for the 

MINIMIZER IN THE MIN-MAX PROBLEM. 






input: a strategy v with N v = 


1 


for i = 1 1 1 


2 


if 3S C V(0), \S\ = i, 4^(0) t Ci(v s (b)) 


3 
4 
5 


Set Vij = b, V{i,j} e $(S) U$*_i (V(v s (b))). 
Exist for loop. 
end 


6 


end 


7 


if N v = 


8 
9 


Set v^ = b for all {i,j} E <$>i (V(0)). 
end 



: {i, j} G £(G)}- Define C n (v) to be a ranking of the set 
C(v) such that C^^\{v) < ... < £<k,\c\(v), and the set 

Ci(v) = $ £ {{{dij + v l3 )fa 3 < C^ i+1 (G) : {ij} G £(G)}) . 

As an abuse of notation, we let C £ n k (v) correspond to the 
value (dij +Vij)faj associated with the link {i,j} G C^(v). 
We assume that C^ x {v) > ... > C^ £ (v). Further, define 
the sets V(v) = {a^faj < : {i,j}£ £i(v)} such that 
Vi(v) < ... < Vyp\(v) and V{v)_ = {fa 3 < : {ij} £ 
£i(v)} such that Vi(v) < ...< V^(v). We also define 



l v s(b)]ij = 



b, {i,j}eS 

0, {iJ}<£S 

The following theorem specifies the optimal strategies of 
both players. 

Theorem 2: The optimal strategy for the adversary in the 
min-max problem is 

f 1, {i,j}eCi(v*) 

u*j(t) = I 0, {i,j}iCi(v*)otf ij >0 

[ {0,1}, otherwise 

For the network designer, the optimal strategy is to run 
Algorithm llj and set v*j = when faj > and v*j G [0, b] if 

fa = 0. 

Proof: The same reasoning as in Thm [T] applies to the 
case when faj > 0. We start by deriving u*. By recalling the 
definition of C^(v*) 9 we can write 



Z^ A ijfij - /_^ A *ijfij + 



j>i 



Aijfa: 






j>i 



< J2 ( a i3+ V tj)fij 






where the last inequality follows from the fact that faj < 
for all {i, j} G C^(v*). This confirms that u* is as claimed. 
Also, note that the last inequality can be achieved because 
|££(v*)| < Z by construction. 

As for the designer, he can now exploit the first-mover- 
advantage since he is acting first. The main goal of the 
designer in this case is to try to protect the links in £^(0). 
These links will be disconnected by the adversary unless the 
designer is able to modify the ranking of the links such that 




Fig. 1. A demonstration of how the designer can alter the ranking of the 
fij's to improve his utility. 



the links (or a subset of them) in £^.(0) are not in C^(v*). 
In essence, this is what Algorithm [I] attempts to achieve. 
This is demonstrated in Fig. [T] Being the lowest negative 
value, and hence the link both the adversary and the designer 
are interested in, let us explore how the designer can push 
C £ n £ (0) higher in the ranking of the link values. The designer 
can achieve this if under some strategy v, the value C £ n £ (0) is 
no longer among the lowest £ negative values; in other words, 
the designer can alter the ranking if there is a set of £ links 
S C V(0) such that when he sets Vij = b for all links in 
<S, there will be £ values that are smaller than C £ n £ (0) (steps 
2 and 3 in Algorithm [I]). The adversary will then break the 
links in S and will spare the link corresponding to C £ n £ (0) 
as required. To see why this is optimal, consider the type of 
links that can be in S. 

• If a link in S is also in £^(0), then this is optimal due 
to the fact that the adversary will disconnect that link 
since it is in £^(0). Hence, if the designer can utilize 
this link to modify the ranking and protect a link that is 
more negative (£^ £ {0) in this case), then this can only 
improve his utility. The same reasoning applies if more 
than one of the links in S are also in £^(0). 

• If none of the links in S is in £^(0), then necessarily 
some of the links in C^ (0) will also be protected along 
with the link corresponding to £^. ^(0). This is because 
1 5 1 = £, and the adversary can only break £ links. 
Hence, this scenario is more favorable to the designer 
than the previous one and can therefore only improve 
his utility. 

If such an S exists, then the designer has exhausted all 
possible moves, since |<S| = £, and the algorithm terminates 
(step 4 of the algorithm). Otherwise, If no such set exists in 
V(0), then the designer should try to protect the next most 
negative link whose value is precisely C £ n i _ 1 (0) by finding 
a set S of of size £-1. Since £^^(0) > £^(0), the link 
corresponding to C £ n £ (0) along with S will constitute the set 
of £ links that the adversary will break. Then, the designer 
should set vij = b for all the links in <S, and for the remaining 
action the designer should select the link with the most 
negative fy that is not in C^(vs(b)); this is precisely the set 
$i (Pfe(t))) (step 3 of the algorithm). The reason behind 
searching in V(vs(b)) and not in V(vs(b)) after finding S 
is that the a^'s only affect the utility of the designer when 
he attempts to alter the ranking. Otherwise, the edge weights 
do not affect the utility of the minimizer as per ([7]). 

This procedure then repeats until the designer has tried 
to protect all the links in £^.(0). If the designer fails in 



protecting all the links in £^(0), then we must have N v = 0, 
i.e., the input strategy was not altered. Then, the optimal 
strategy is to set Vij = b for the links with most negative 
fij's in V(0) (steps 7 and 8 in Algorithm [j}. ■ 

C. Complexity of the Optimal Strategies 

Let us now study the complexity of the optimal strategies. 
We first start by the max-min problem. Assuming as in 
Remark [T] that the players switch their strategies a total of K 
times over [0, T], we conclude that the worst-case complexity 
of the strategy of both players is 0(K • mlogm) as their 
strategies involve merely the ranking of sets of size at most 
2m. As for the min-max problem, the complexity of the 
adversary's strategy is 0(K -m log m). The main bottleneck 
in the strategy of the designer is step 2 in Algorithm [I] The 
size of the set V(0) is at most m — £\ thus, the worst- 
case complexity for the designer is K • J^te^i ( m *~ ) ~ 
K • ^2 i=1 (m — £)\ By comparison with d2b, we conclude 
that the derived optimal strategies achieve vast complexity 
reductions. 

IV. Implementing the Optimal Strategies 

The optimal strategies we derived in Section [III] are defined 
in terms fij's which depend on the state x(t) and the costate 
p(t). However, we have not derived the optimal trajectories 
that satisfy the canonical equations given by the MP in ^ 
and ([5]). Since the system is linear time- varying, the solutions 
will be given in terms of a state transition matrix; this makes 
working with fy intractable. The following theorem provides 
a procedure to arrive at the optimal solutions without the 
need to compute p(t). Parts of the proof of this theorem 
appeared in [12]. Define v\j = — (xi — Xj) 2 . 

Theorem 3: The rankings performed as part of the opti- 
mal strategies of the max-min and min-max problems can 
be carried out by replacing fij{t) by v%j(t). Further, it is 
optimal for the players to modify a total of £ links. 

Proof: We will provide the main ideas behind proving 
this claim for the max-min problem - the same arguments 
will apply to the min-max problem. For a fixed strategy of 
the adversary u* 9 we will show that it is optimal for the 
minimizer to rank the links based on their ^ values instead 
of the fij's. Then, by applying the same reasoning as in Thm 
[T] we arrive at u* with the ranking performed using u^. 

The main complication in solving the adjoint equations is 
that the system is time- varying. However, since x(t),p(t) G 
C 1 [0, T], the value of fij cannot change abruptly in a finite 
interval. As a result, the control obtained from the MP cannot 
switch infinitely many times in a finite interval. Hence, we 
can regard the system as a time-invariant one over a small 
interval [to, to + 8] C [0,T], S > 0. The proof consists of 
three steps. 

1. Show that it is optimal for the players to change £ links. 

2. Show that, over a small interval [to, to + 8], it is optimal 
for the designer to switch from a strategy v A to a 
strategy v B that entails ranking the links based on their 
v^ values. 



3. Show that mimicking v A for the remaining time of the 
problem preserves the gain obtained over [to , to + S] . 
The first step was proven in [12] for the maximization 
problem. In the min-max (or max-min) case, the proof 
remains the same. This is because for a fixed u*, then by 
Thm 1 in [12], it follows that it is optimal for the designer 
to modify £ linksQ By Thm[l] we deduce that N u * = L 

We want to show that the designer will modify the £ links 
with the lowest v^ values. Assume that the designer applies 
a stationary strategy v A over the interval [to , to + 5] with a 
corresponding system matrix A. Let P(t) := e At . Due to 
the structure of A, P(t) is a doubly stochastic matrix for 
t > 0. Because the control strategies are time-invariant, the 
state trajectory is given by 

3 A ^-^x(to), te [t ,t + 6\. 



i 



x(t) 

Assume that the links the designer modifies over this interval 
are not the ones with the lowest v^ values. In particular, 

- and v B t = b although 



assume that the designer sets v^ 



Vij < Vki- At time t* G [to, to + 6], we assume that the 
designer switches to strategy v B by setting v B = b and v^ = 
0. Let the matrix B be the system matrix corresponding to 
v B , and define the doubly stochastic matrix Q(t) := e Bt , 
t > 0. Showing that this switch can improve the utility of 
the designer is equivalent to proving the following inequality 
[12]: 

rto+S 

k(t) ■ x(t ) T A(t, t*)x(t )dt < 0, (8) 



where A(t,t*) = P(t* -t )Q(2(t-t*))P(t* -t )- P(2(t- 
to)). A sufficient condition for (J8J) to hold is 

h(t,x(to)) = x(t ) T A(t,t' k )x(to) < 0, for t > t*. 

As t I 0, we can write P(t) = I + tA + O (t 2 ), where 
O (t 2 ) jt < L for some finite constant L. Following similar 
steps to those in [12], we can write 

h(t, X(t )) = 2(t - t*) J2( A sr ~ B sr) (x r (t ) ~ X 8 (t )f 

r>s 

= 2b(t - t*)(^(t ) - m(to)) + 0(S 2 ). 

For small enough S, the higher order terms are dominated 
by the first term. Hence, if there are links {i, j} and {fc, /} 
such that Uij < v^i, there exists t* such that h(t,x(to)) < 
for t G (t* , to + 5] . This proves that the optimal strategy for 
the designer is to break the links with the lowest Vij values. 
By comparison with the statement of Thm [TJ we conclude 
that we can replace fy by v^ while performing the required 
rankings. 

The final step of the proof is to show that switching to 
strategy B guarantees an improved utility for the adversary 
regardless of how the original trajectory A changes beyond 
time to + S. To this end, we will assume that from time 
to + 5 onward, strategy B will mimic strategy A. Assume 
that strategy A switches to another strategy C starting at time 

Conditions were given in [12] as to when it is also optimal for a player 
to change less than £ links. 




Fig. 2. 
proof. 



r 

A(u^,v 2 ) 

A demonstration of the technique used in the third step of the 



to + S. Hence, strategy B will also switch to strategy C, but 
the trajectories will have a different initial conditions. Fig. [2] 
illustrates this idea. Consider the behavior of the system over 
[to + 5, to + 2 S] over which we can assume that the system 
is time-invariant. One can show that it suffices to prove that 
[17] 

r^o+25 

k(t) • c(t)dt < 0, 

lto+5 

where c(t) is the difference between the utilities resulting 
from the trajectory that includes a switch to strategy B at 
time t* and the one that does not switch at time t*. After 
performing a first-order Taylor expansion, c(t) can be shown 
to be equal to [17] 

c(t) = 2b (to + S - t*) (z^(t ) - v ki (to)) + 0(5 2 ), 

which is negative for small 5. Hence, the gain that was 
obtained by switching to strategy B at time t* is preserved 
and the proof is complete. ■ 

Remark 3: (Potential-Theoretic Analogy) As discussed in 
[12], when the graph is viewed as an electrical network, 
ciij+Vij can be viewed as the conductance of link {i, j} and 
Xi — Xj as the potential difference across the link. Therefore, 
the optimal strategy of the designer in both problems involves 
finding the links with the highest potential difference (or 
the lowest z^/s) and increasing the conductance of those 
links by setting v^ = b. This leads to increasing the 
power dissipation across those links, which translates to 
increasing the information flow across the network and 
results in faster convergence. The optimal strategy of the 
adversary should therefore involve breaking the links with 
highest power dissipation. But power dissipation is given by 
(aij + Vij)(xi — Xj) 2 , and this is exactly what the adversary 
targets according to Thm [3] 

V. A Sufficient Condition for the Existence of 
anSPE 

Having solved the min-max and max-min problems, it 
remains to verify whether the value of these problems can 



be equal, even though the strategy sets of the players are 
non-rectangular. The reason that the upper and lower values 
are different is mainly due to the ability of the minimizer 
to deceive the maximizer by altering the ranking of the 
most negative values. If we remove this ability from the 
network designer, we should expect that an SPE would exist. 
The following theorem makes this argument formal. Define 
7 := ff , where M, e are defined in Lemmas fil and in the 
Appendix. We assume that e is chosen to guarantee 7 > 1. 
Theorem 4: A sufficient condition for the upper and 
lower values V,V_, defined in ^ to be equal, and hence 
for the existence of an SPE, is to select b such that 



< b < min han 

{ij},{k,i}es l J 



Q>kl\ 



(9) 



given that a^ ^ a k \ and a^ > ja k i whenever a^ > a k u f° r 
all{U},{fc,Z}e£. 

Proof: It suffices to show that Ci(v*) = ££(0) = 
D^(G*) as this would imply that the adversary would break 
the same links whether he acts first or second, and as a result 
the strategy of the minimizer in both problems will be the 
same. This would occur if the minimizer cannot protect any 
of the links in £^.(0). In other words, this will occur if the 
minimizer cannot satisfy the condition in step 2 of Algorithm 
[I] for any 1 < i < L A sufficient condition for this to happen 
is to require 



min (an 

{iJ}ev(oy 



b)fij > 



{ij}eci(o) 



j Jij • 



This implies that no matter how the designer changes the 
weights of the links in V(0), it cannot make those links more 
negative than the links in £^.(0). To satisfy this inequality, 
we will establish that whenever a^j^ > a H f kU fijjki < 0, 
we must have (a^ + b)fij > a k if k i, for all links in S. We 
can then re-write the condition on b as 



b< 



a ijfij ~ a klfkl 
~Jij 



O'kl 



\fi 



kl\ 



\k 



(10) 



Consider the following two cases. If f k \ > fij, then we 
must have a k \ > a^. Then, by assumption we have that 
dki > jaij . By Lemmas [T] and 151 we can write 



O'kl 



\fi 



kl\ 



\k 



Oij > 



-Okl 



7 



dij > 0. 



(11) 



Next, consider the case when fy > f k \. In this case, a^ can 
be greater or smaller than a k \. However, if a^ > a k i, and 
recalling that a^fy > a k ifku then 



\fi 



kl\ 



ja M < dij < o k iff^ < ja kh 

I Jij I 

which is a contradiction. The case a k i = a^ is excluded by 
assumption. Hence, in this case, we must have a^ < a k u 



and the inequality in \\A) applies. Thus, by choosing b as in 
([9]), we obtain the condition we are seeking. Note that we do 
not need to consider the case when a^fij — a k \j k \ since the 
players will be indifferent as to which link to choose when 
the corresponding values are equal. ■ 

Remark 4: The condition derived in the above theorem 
requires the network to be sufficiently diverse in the sense 



that the weights of the links have to be not only different 
from each other, but also a factor 7 apart. This is due to the 
fact that we were seeking uniform bounds on the fij's. If we 
allow b to vary with time, then one can find less restrictive 
conditions to ensure the existence of an SPE. However, this 



would require ( 10) to be verified at each time instant. Further, 
the bounds derived in the Appendix are loose. Tighter bounds 
could be given by studying the dynamics of \f k i\/\fij\. 
However, studying the time derivative of this ratio is not 
tractable. 

VI. Conclusion 

In this paper, we have formulated and solved two problems 
that capture the competition between an adversary and a 
network designer in a dynamical network. We considered 
practical models for the players by constraining their actions 
along the problem horizon. The derived strategies were 
shown to exhibit a low worst-case complexity. We also 
proved that the optimal strategies admit a potential-theoretic 
analogy, and they can be implemented without needing to 
solve the adjoint equations. Finally, we showed that when 
the link weights are sufficiently diverse an SPE exists for 
the zero- sum game between the designer and the adversary. 
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Finally, recall that | \x(0) \ \ 1 < ft- 1 1#(0) 1 1^. We therefore have 
\fij\ = \Pi-Pj\\xi-Xj\ < 4nT(||x(0)|| oo + Wv 9 \) IkWIL , 



Lemma 1: The 

from above by 



Appendix 
function fij{t) 



is uniformly bounded 



M:=4nT(||x(0)|| c 



°avg | 



ll*(o)ll c 



as claimed. ■ 

Lemma 2: Given 5, e > 0, one can select the problem 
horizon T small enough such that \fcj \ > e 2 for all t G [0, T]. 
Proof: We now show that \f^ (t) | can be bounded below 
uniformly by a positive number. In order to do so, we need to 
ensure that \xi(t) —Xj(t)\ does not approach zero as t — » T. 



Proof: By the structure of the system matrix in {j}, we We are seeking a time t* such that for a given e > 0, we have 



can deduce that \x{ 

\Xi(t) - Xj (t)\ < 



cannot increase as t — >> T. Thus 



< 



max 

l<i,j'<n 

2 max 

Ki<n 



\ Xi (0) - Xj (0)\ 
\ Xi (0)\ = 2 ||x(0)|| c 



In order to bound \pi (t) —pj (t) | , we consider the interval [T— 
5, T] where S > is small enough such that the system in 
([T]) is time-invariant. Let the system matrix over this interval 
be A and define s := T — S. The solution to the adjoint 
dynamics in ^Q, ^ over [s, T] is then given by 

3 A ^X(S) 

2 I e- A ^- T Hx(r) - x)dr. 



x(t) = 

pit) = 2 /" e- A ^- T \x(r) 
Define the doubly stochastic matrix P(t) : 



(12) 
(13) 



G At 



Then 



|Pi - Pi I 



<2 



<2 



<2 



E 
* it=i 

T 



(Pi(T -t)- Pj(t - t))(x(r) - x)dr 



\{P ik {T -t)~ P jk (T - t))( Xk (T) - X k )\dT 



Jt fe=i 



• x fe I dr 



x(r)\\ 1 +n\x avg \)d,T. 



< 



Note that over [s,T\ we have | |rz?(*) 1 1 x = \\P(t - s)x(s 
\\P(t — s)\\ 1 ^(s)!^ = ||^(s)||i, where the last inequality 
follows from the fact that P(t) is a stochastic matrix. By 
repeating the above argument over the interval [0, s), after 
dividing it into smaller intervals and exploiting the fact that 
the solution over each small interval will be of the same 
form as in (12), we conclude that 11^(^)1^ < 11^(0)1^, for 
all t G [0, TjTWe can therefore write 

\ Pi (t)- Pj (t)\ <2<J(||a?(0)|| 1 + n|a; o ^|), t G [s,T\. 

When we propagate the solution backward over the interval 
[T— 25, 5], a new integral term will be added to the expression 
in fl3] ); however, it will be of the same structure as that of 
pit) over [s, T]\ an integral of a doubly stochastic matrix that 
depends on the system matrix over [T — 25, T — 5] multiplied 
by x(t) — x. This applies for all infinitesimal intervals we 
consider from T — 25 down to zero. Hence, if we subdivide 
[0, T] into K intervals of length 6 and apply the triangular 
inequality to \pi(t) —Pj(t)\, we will obtain 

\Pi(t)- Pj (t)\ < 2T(||x(0)|| 1 +n\x avg \) , t G [0,T]. 



(t) — Xj (t)\ > e for all t < t* and all z, j. We can then fix 
T < t* to ensure the existence of a uniform lower bound on 
\xi(t) — Xj(t)\. Let us again restrict our attention to a small 
interval [to , to + 5] where the system is time-invariant, and 
let the system matrix over this interval be A. We require that 
the system did not reach equilibrium over this interval, i.e., 
x(to -\- S) ^ x. Without loss of generality, we assume that 
Xi(to) > . . . > ^ n (^o)|5 D e fi ne the following dynamics 

liVi-x^to)) = ^Aijfaito)-^), 

;(y,-Xn(to)) = ^Aijtxni^-y.), 



dt 



dt 



with initial conditions y^to) — 2x\(t§), y .(to) = 2x n (to). 
Note that x\ = J2j^i^ij( x j ~ x i)- It follows that y. < 
Xi <Vi> By the comparison principle, we conclude that y . — 
x n (to) ^ Xi <y i — xi(to), for 1 < i < n. Note that we can 
readily find the solution trajectories for y and y. By defining 
a i = ^j^i Aij, we can then write 

Vi-x^to) = e-^-^x^to), 

y.-x n (t ) = e- a ^-^x n (to). 

By solving the equation y i _ i —x\{to) — y. — x n (to), we can 
find a time t\ when x^_i can potentially meet xf. 



t* 



1 



In 



xi(t ) 



t . 



CLi-i-CLi \X n (t ) / 

If t\ > to +5, for all z, then we need to propagate the solution 
forward, and keeping in mind that the system matrix could 
change, until we find a time t\ in some interval [£, i + S] 
where y i _ i = y. for some i. Then, for a given e > 0, we can 
select T < t* such that \xi — Xi-i\ >\y. — Vi-i\ > e; hence, 
we conclude that for this choice of T we can guarantee that 
\xi — Xj\ > e > for all z, j. 

Since p(T) = 0, then \p { - pj | ->> as t -> T. Given 
e > 0, there is a time T{j (e) < T such that \pi — Pj\ < e, 
for all t > fij(e). Define the time f= min T^-(e). Then 

l<i,j'<n 

\Pi — Pj\ > e for t > T(e), for all z, j. We therefore have 

\fij\ = \pi - Pj\\xi - Xj\ > e 2 > 0. ■ 



2 We are making the implicit assumption that sci(0) > ... > x n (0). 



